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Abstract — This paper proposes a distributed framework for 
demand response and user adaptation in smart grid networks. 
In particular, we borrow the concept of congestion pricing in 
Internet traffic control and show that pricing information is very 
useful to regulate user demand and hence balance network load. 
User preference is modeled as a willingness to pay parameter 
which can be seen as an indicator of differential quality of service. 
Both analysis and simulation results are presented to demonstrate 
the dynamics and convergence behavior of the algorithm. 

Index Terms — Smart grid, demand response, pricing, utility. 



I. Introduction 

A smart grid is an intelligent electricity network that in- 
tegrates the actions of all users connected to it and makes 
use of advanced information, control, and communication 
technologies to save energy, reduce cost and increase relia- 
bility and transparency. Recently, many countries have started 
massive efforts on research and developing smart grids. For 
example, the smart grid is a vital component of President 
Obama's comprehensive energy plan: the American Recovery 
and Investment Act includes 11 billion USD in investments 
to "jump start the transformation to a bigger, better, smarter 
grid". 

In electricity grids, demand response (DR) is a mechanism 
for achieving energy efficiency through managing customer 
consumption of electricity in response to supply conditions, 
e.g., having end users reduce their demand at critical times 
or in response to market prices. In the future smart grid, the 
two way communications between energy provider and end 
users enabled by advanced communication infrastructure (e.g., 
wireless sensor networks and power line communications) and 
protocols will greatly enhance demand response capabilities 
of the whole system. In contrast to the current simple time- 
of-use (TOU) pricing (e.g. peak time vs. off-peak time), it 
can be envisaged that a more dynamic, real-time adaptation 
to market prices would not only enable consumers to save 
more energy and money, as well as manage their usage pref- 
erences more flexibly, but also facilitate the grid move closer 
towards its optimal operating point. For a recent overview of 
challenges and issues of enabling communication technologies 
in this area, please refer to [1 1. The authors of |2| argue that 
demand response and distributed energy storage can be seen as 
distributed energy resources and are main drivers of smart grid. 
While DR can help the industry to achieve market efficiency 
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and operational reliability, there are also challenges ahead in 
implementing DR under smart grid and market paradigms. 

There are a few papers recently on smart grid DR using load 
scheduling. In f3l, user preferences are taken into account with 
the concept of discomfort level and an optimization problem 
is formulated to balance the load and minimize the user in- 
convenience caused by demand scheduling. Several ideas from 
the distributed computing area such as makespan have been 
introduced to energy consumption optimization. Similarly, in 
im, an energy consumption scheduling problem is established 
to minimize the overall energy cost. Techniques similar to 
those used in wireless network resource allocation have been 
applied here to solve the underlying optimization problem. In 
both works, the user demands are known beforehand and the 
optimization problem is solved in numerical iterations. 

In this paper, we consider a fully distributed system where 
the only information available to the end users is the current 
price which is dependent on the overall system load. Based on 
this information, the users try to adapt their demands so as to 
maximize their own utility. There is no central control entity. 
Inspired by the well-established work on congestion pricing in 
IP networks, we propose a simple adaptation strategy based on 
price feedbacks and show that it is very effective in achieving 
demand response. 

The rest of the paper is organized as follows. Section 2 
introduces our DR model and the adaptation algorithm. We 
present some simulation results in Section 3. Conclusions and 
future work are presented in Section 4. 

II. Demand response model 

A. Congestion pricing background 

In this paper we propose to apply the principle of congestion 
pricing in IP networks to demand response in the electricity 
grid. In their seminal paper |5|, Kelly et al. have proposed 
the proportionally fair pricing (PFP) scheme in which each 
user declares a price per unit time that he is willing to pay 
for his flow. In that sense the network capacity is shared 
among the flows of all users in proportion to the prices paid 
by the users. It has been shown in |5| that in a weighted 
proportionally fair system where the weights are the prices the 
users pay per unit time, when each user chooses the price that 
maximizes the utility she gets from the network, the system 
converges to a state where the total utility of the network is 
maximized. In other words, in an ideal environment, the PFP 
proposal is able to decentralize the global optimal allocation 
of congestible resources. Another important result of ||5] is 
that rate control (such as TCP) based on additive increase and 



multiplicative decrease achieves proportional fairness. It has 
been proved in |6| that the decentralized congestion control 
mechanism is stable even under arbitrary network topologies 
and heterogeneous round trip times (feedback delays). 

In Kelly's approach, the philosophy is that users who are 
willing to pay more should get more. As the network makes 
no explicit promises to the user, there is no need for over 
provisioning in the core of the network. One implementation 
of PFP is to give control to end systems (users). In this scheme, 
the TCP algorithm is modified to incorporate congestion prices 
by means of protocols like explicit congestion notification 
(ECN) Q. Upon receiving feedback signals, f{t), which are 
related to shadow prices (in terms of packet marks), the users 
are free to react as they choose, but will incur charges when 
resources are congested. An end system can adjust its rate x{t) 
using a willingness to pay (WTP) parameter w: 



^.{t + l) = x{t)+a{w~f{t)), 



(1) 



where a affects the rate of convergence of the algorithm. 

In fS], explicit prices instead of marks are fed back to the 
end users as incentives and users adapt their rates accordingly. 
It has been shown that the system converges to an optimal 
allocation of bandwidth: the users' price predictions converge 
to the actual price and their bandwidth allocations converge 
to levels which equalize their marginal utility of bandwidth to 
the price of bandwidth. 

B. The DR model and user adaptation 

We consider a discrete time slot system where N users 
share some energy resources. In each time slot n, user i has a 
demand of Xi{n) (e.g. hourly energy consumption if the time 
granularity is one hour). The unit price of energy in a time 
slot is a function of the aggregate demand: 



N 



P{n) ^ f(y x,{n)) 



i=l 



(2) 



The price function (spot market price) can be of the following 
form mmi: 

f{x)=a{^)\ (3) 

where a and k are constants, and C is the capacity of the 
market. 

Each user i is associated with a utility function Ui{xi{n)) 
in time slot n, which is a concave, non-decreasing function of 
its demand. A typical logarithmic utility function is given by 
0: 



Ut{x) = Wi log X, 



(4) 



where Wi is the willingness to pay parameter. Hence user i 
chooses its demand Xi{n) to seek to maximize 



Ui{xi{n)) - Xi{n)p{n). 



(5) 



We would like to elaborate on a few assumptions made 
in the above model. Firstly, in our model the demand a; is a 
continuous variable, which may not be realistic in practice. For 
example, in real life, the daily usages of a washing machine 
and a dryer are (fixed as) 1 .49 kWh and 2.50 kWh respectively. 



Here the adaptation of x can be seen as an action of load 
scheduling: for example, re-scheduling a dryer operation from 
time slot n = 1 to slot n = 3 leads to x{l) reduced by 
2.50 kW per hour and a::(3) increased by 2.50 kW per hour 
Secondly, how to characterize user preference is an open issue. 
For instance, a user may prefer his washing done at 6pm which 
is a typical peak time. To some extent, this preference can be 
reflected in the WTP parameter w in (|4]i: when a user is willing 
to pay more, he/she can have a higher demand. However, the 
delay (or waiting time) incurred due to rescheduling is not 
considered in this model. Thirdly, as pointed out in |5|, log- 
arithmic utility functions lead to proportional fairness. There 
are other types of utility functions available corresponding to 

Wi^^^.P < 1 as 



different fairness criteria, e.g. Ui{x) — w 
proposed in [8|. How to choose a most suitable utility function 
in DR applications is an open issue, e.g. how to factor in the 
waiting time and user discomfort level f3l. 

User i adapts its demand according to the following equa- 
tion: 

Xi{n + 1) = Xi{n) + ai{w^ - Xi{n)p{n)), (6) 

where ai is a parameter that controls the rate of convergence 
of the algorithm. It is clear that the user adjusts her demand 
according to the price information (p{n)) and her own will- 
ingness to pay preference (w). 

To show that the above adaptation converges to the user 
optimum, let us assume that the equilibrium price is q. Then 
by solving u^{xi{n)) = q, we have the optimal demand x* as 

X, = —. 

q 

Given (|6]l, the error of demand estimate, ei{n + 1), is given 
by 

iv ■ 

ei{n + l) = Xi{n + 1) ~ x* ^ Xi{n) + ai{wi ~ Xi{n)q) -. 

h 

Then it follows that 

ej(n + 1) = (1 - atq){xi{n) -) = (1 - aiq)ei{n). (9) 

Therefore ei{n) is a geometric series, and when |1 — aiq\ < 
1 , lim„_j.oo £i (n) — 0. This has established that with properly 
chosen ai, the adaptation will converge to the optimum. 
Following Q, it is also straightforward to establish the global 
stability of the algorithm in a differential equation form (fTOl) 
using an appropriate Lyapunov function. 



(7) 



-Tf^i{i) = oii{wi - Xi{t)p{t)). 



(10) 



C. Implementation considerations 

In a residential energy management scenario, we envisage 
that each user in our model is represented by an entity or 
software agent called home energy manager (HEM) at a 
consumer's home. Appliances in the home are equipped with 
smart meters, and they communicate with HEM via low power 
wireless such as ZigBee. HEM is further connected to the grid 
(supplier) via either wired or wireless links. Based on the price 
information it receives, HEM calculates demand in the next 





Fig. 2. Simulation IIII-AI demand adaptation of 10 users 



Fig. 1. Distributed demand response 



time slot and distributes it to different appliances. The overall 
architecture is shown in Figure [T] 

We note that some appliances like refrigerator and heating 
have hard consumption scheduling requirements, while others 
such as washing machine have soft requirements |4|. When 
HEM has to shift the demand to another time slot, it may 
apply only to soft appliances. For example, HEM obtains 
x{n + 1) based on ^ with WTP parameter w. If it can 
satisfy the demand from the hard appliances (denoted by h), it 
can re-schedule the demand from some of the soft appliances 
(denoted by s) so that x{n + 1) = h{n + 1) + s{n + 1). 
On the other hand, if it cannot meet the demand from the 
hard appliances, HEM may have to increase w and recalculate 
x{n + l). 

III. Simulation results 

In this section, we use simulations to study the behavior and 
dynamics of the proposed algorithm. There are A^ = 10 users 
and without loss of generality we assume that the capacity C 
is 1. For the price function (O, a = 1, fc = 4. 

A. Basic simulation 

Here all the users initiate their demands at 0.02, and their 
willingness to pay parameters range from 0.1 1 (user 1) to 0.20 
(user 10). All the users have the same adaptation parameter 
a of 0. 1 . Figure |2] shows the demand changes with time for 
10 users. After a short transient period, each user demand 
converges to a stable value (determined by different w values). 
It is also evident that w is a crucial factor in determining how 
aggressive a user should be responding to the price signals. 
Figure [5] clearly shows that the price converges to the optimal 
value. When the system reaches its equilibrium (assuming a = 
l,C = 1), we have 




Fig. 3. Simulation IIII-AI price evolution 

Summing over i on both sides of ( fTTT ). it is easy to verify that 
the price at equilibrium is 



N 



P 



(^W.H)^! 



(12) 



In this case, p = 1.42 as shown in Figure [3] 



Xi 



-N 



{Eti^dn)y 



(11) 



B. The effect of a 

In this simulation experiment we study the effect of a on 
system performance. Figure |4] and Figure |5] depict the demand 
and price evolution versus time respectively for a = 0.17. 
Compared with Figure |2] and Figure [3] it can be seen that with 
a larger a, it takes much longer to converge. Therefore a is 
an important system parameter that controls the convergence 
speed of the process. 

C. Heterogeneous initial demands 

In this simulation experiment we study the effect of hetero- 
geneity of initial demands, i.e., ten users start with demands 
ranging from 0.01 to 0.10 respectively. The results are shown 
in Figure |6] and Figure |2l We observe that different initial 
conditions do not affect the system stability and convergence 
to equilibrium. 




Fig. 4. Simulation IIII-B I demand adaptation of 10 users 
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Fig. 5. Simulation IIII-B I price evolution 
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Fig. 6. Simulation IIII-CI demand adaptation of 10 users 




Fig. 7. Simulation IIII-CI price evolution 




Fig. 8. Simulation IIII-DI demand adaptation of 10 users 



D. Heterogeneous initial demands and adaptation rates 

In addition to heterogeneous initial demands as in last 
simulation, here users also have different adaptation rates ac 
ranging from 0.11 to 0.20. The results are shown in Figure |8] 
and Figure |9] where we can see that the system still converges 
to the equilibrium. 



E. Time-varying w 

To model the situation where users change their WTP w on- 
the-fly to accommodate their energy needs, we change w/s at 
time slot 100 by adding a random number within the region 
of (—0.05,0.05). Figure [To] and Figure [TT] clearly show that 
after time 100, the system tracks the change nicely to a new 
equilibrium. 



F. The effect of C 

The energy provider can influence the price by adjusting 
the capacity C As shown in Figure [12] and Figure [T3] when 
C is doubled, the price will drop by 1 — C^+r which is 43%, 
and each user's demand will increase accordingly. 
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Fig. 9. Simulation IIII-DI price evolution 





Fig. 12. Simulation IIII-FI demand adaptation of 10 users 




Fig. 13. Simulation IIII-FI price evolution 
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Fig. 10. Simulation IIII-EI demand adaptation of 10 users 




Fig. 11. Simulation IIII-EI price evolution 



G. Inaccurate price signals 

The feedback price signals are transmitted via a communi- 
cation network (e.g. GPRS) to the HEM, during which packet 
loss and delay could occur. In this case users may have to 
adapt their demands based on outdated or inaccurate price 
information. We model this situation as a small perturbation 
to the price signal and study its effect on the system behavior 
As shown in Figure [14] and Figure [15] the price and demands 
still converge to the means of the equilibrium values, but with 
small fluctuations. A more detailed perturbation analysis is 
part of our future work. 

IV. Conclusion and future work 

This paper proposes a distributed framework for demand 
response and user adaptation in smart grid networks. More 
specifically, we have applied the concept of congestion pricing 
in Internet traffic control to the DR problem and shown that it 
is possible that the burden of load leveling can be shifted from 
the grid (or supplier) to end users via pricing. Individual users 
adapt to the price signals to maximize their own benefits. User 
preference is modeled as a willingness to pay parameter which 
can be seen as an indicator of differential quality of service. 
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Fig. 14. Simulation IIII-GI demand adaptation of 10 users 




Fig. 15. Simulation Illl-GI price evolution 



The convergence of the algorithm has been demonstrated by 
both analysis and simulation results. 

This paper is just a first step towards our vision of fully 
distributed demand response. There are a number of directions 
for future research. Firstly, the proposed model fits nicely into 
the game theory framework. In fact, there is already a rich 
literature in the networking community on game theoretical 
analysis of congestion pricing, e.g. 1 10|. In Kelly's framework, 
the user and social optima coincide if the prices are right, 
and the social optimum is a Nash bargaining solution with 
the logarithmic utility function. More recently, researchers 
have applied intelligent agents and game theory to micro- 
storage management for the smart grid 111], in which Nash 
equilibrium is reached when the agents are able to optimize 
the energy usage and storage profile of the dwelling and learn 
the best storage profile given market prices at any particular 
time. Similarly, based on our model, it would be interesting 
to study the system dynamics and user interaction in a large 
scale energy demand game context. There are two types of 
user strategies: price taking users and price anticipating users 
lfT2J . A price taking user assumes that he has no effect on the 
price of the energy, whereas a price anticipating user realizes 
that his own choice of Wi affects the price. An interesting 



observation is that price anticipating users tend to pay less. 

One important element of intelligence in the smart grid 
is the learning capability of various components. In demand 
response, if users can learn from past observations (e.g. prices 
and load profiles), then they can predict the future load and 
price and adjust their strategies accordingly (e.g. adjusting a^ 
and Wi). In this context, Bayesian networks and reinforcement 
learning are some of the powerful tools we can leverage to 
enable learning in this highly dynamic environment. 

As mentioned earUer, demand scheduling can be formulated 
as a typical resource allocation problem, for which a wide 
range of techniques (many of them have been applied in 
the networking field) are available. We are currently inves- 
tigating the feasibility of applying some convex optimization 
techniques such as water-filling to demand side management, 
where a certain cost function is to be minimized subject to a 
number of system constraints. 
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